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implemented. While teachers reported high levels of awareness of standards, 
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district accountability policies that were narrowly focused on raising 
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Influence of State Policy on Standards and School Practices: 

A Comparison of Three Urban Districts 

This research examined the implementation of standards-based reforms in urban schools 
with high concentrations of low-income students. In particular, we were interested in the 
influence of the state policy on the development and implementation of school standards. We 
examined the type of state policy three school districts were exposed to, the level of teachers’ 
awareness of the state policy on standards, and teachers’ perceptions of the implementation of 
standards in schools. Our goal was to understand the role of standards as a driving force in 
accountability frameworks. 

We focused on Title I schools with schoolwide programs because the federal program 
now includes the goal of helping students who attend high-poverty schools meet high standards. 
The 1994 Improving America’s Schools Act (IASA), which reauthorized Title I of the Elementary 
and Secondary Education Act, represented a shift in the program’s strategy for helping children in 
high poverty schools. Instead of a program focused on remediation, the program aims to improve 
the quality of education children receive (Wang, Wong, & Kim, 1999; Vinovskis, 1999; Kim, 
1998). This legislation mandated that rigorous national standards apply to all students, including 
those receiving Title I services and requires states to develop content and performance standards 
to qualify for Title I funding. In addition, states are required to adopt student assessment systems 
aligned with the standards and use these assessments as a criteria to determine the “adequate 
yearly progress” of schools and districts towards meeting the goal of improving student 
performance. Both states and districts are required to develop a plan that incorporates ways to 
help schools meet the state’s student performance standards. Since the legislation is non- 
prescriptive, describing only general expectations, states have considerable discretion regarding 
the content of standards and assessments and how to help students meet the higher standards. 

This research took the perspective that the context of school reform is important in 
understanding how schools and teachers implement standards. In particular, district responses to 
state policy are likely to have implications for how schools allocate resources (Bidwell & 

Kasarda, 1980; Barr & Dreeben, 1983). Under the federal IASA Title I legislation, districts must 
interpret and implement state level standards and assessments, develop criteria to identify schools 
in need of improvement, and provide assistance to schools identified as in need of improvement. 
They can take corrective actions when schools fail to improve after three years. In addition, 
states have enacted other legislation that has expanded the managerial and administrative role of 
districts to intervene in low performing schools. In other words, district policies and practices 
mediate between state actions and the schools. 

This paper is organized as follows. In the following section we describe the study’s 
research design. The third section presents the policy context of standards-based reform. We 
discuss the accountability systems that have emerged in each of the three districts in the study, 
highlighting the policies and strategies adopted in each district to improve academic performance 
and support schools in their efforts. The fourth section presents the results from a teacher survey 
that measured teachers’ awareness of state standards policies and their perceptions of how well 
schools are implementing them. The concluding section of the paper includes an exploratory 
discussion of the implications of the findings on the design and implementation of standards- 
based policies and on schooling for disadvantaged students. 




3 



i 



Research Design 



This research examined the influence of state policy on teachers’ awareness of standards 
in urban districts with high concentrations of low-income students. It addressed three questions. 
What are the state and district policies on standards? How are these policies related to the level 
of teachers’ awareness and perceptions of state standards? What is the relationship between 
teachers’ perceptions of state standards and school practices? 

We selected three urban districts — Cleveland, Chicago, and Detroit — for inclusion in this 
study. 1 These districts were chosen because they represented urban school districts in 
Midwestern states with high concentrations of low-income students. 2 Using a multiple regression 
analysis, we identified four schools in each district for inclusion in the study. This analysis 
examined the relationship between average levels of student achievement in math and reading 
and demographic characteristics and designation as magnet schools (Yancey & Thadani, 1997; 
Yancey, Freely, & Bredding, 1999). 3 Each school received an expected score, that is, the level of 
achievement that would be expected given the characteristics of specific schools. This expected 
score was then compared to the actual achievement score, indicating whether or not the school 
was performing above or below expectations. Schools with average achievement scores that were 
consistently above or below what was expected were identified as “exceptional.” A sample of 
four schools was selected in each district that included two schools from each end of the 
1 distribution (table l). 4 All schools qualified for the Title I schoolwide program. The final site 
selection took into consideration the racial composition of the schools to insure the inclusion of 
one school in each district with a predominately Latino student population (see table 2 for the 
background characteristics of each school). In addition, site selection was influenced by forced 
substitutions when schools declined to participate. In Detroit, district administrators selected the 
- schools from a list of schools at each end of the distribution. In Cleveland and Chicago, 
researchers made the selection. 

[INSERT TABLE 1 ABOUT HERE] 

[INSERT TABLE 2 ABOUT HERE] 

Data collection took place during the 1997-98 and 1998-99 school years. We collected 
data using multiple methods, including surveys, interviews, observations, and document analysis. 



1 This study was part of a larger study on the implementation of Title I schoolwide programs sponsored by 
the Laboratory Network Program of the Regional Educational Laboratories (Wang, Wong, & Kim, 1999). 
The larger study, a collaborative project among six Regional Educational Laboratories, was designed to 
contribute to a national database on the implementation and outcomes of Title I schoolwide programs since 
the enactment of IASA in 1994. Districts selected for this larger project were in the region served by each 

three of the seven states served by the North Central Regional Educational 
Laboratory (NCREL), one of the laboratories participating in the larger study. The seven states served by 
NCREL included Illinois, Indiana, Iowa, Michigan, Minnesota, Ohio, and Wisconsin. 

3 The socioeconomic variables included percent of students who qualified for free or reduced price lunch, 
the stability of the student population over the academic year, the percent of students who were African 
American, Latino, Asian or American Indian, the percent of students with limited English proficiency, the 
average daily attendance, the size of the school, and whether or not the school had a special program 
designation, such as magnet school. The analysis included data for seven years, 1990 to 1996, in Chicago; 
two years, 1995 and 1996, in Cleveland; and one year, 1996, in Detroit. See Yancey and Thadani, (1997) 
for more on methodology. 

4 School names are pseudonyms.. 



participating laboratory. 
2 These districts were in 
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In order to understand the influence of state and district policy on the implementation of 
standards, we first sought to understand how the accountability system worked in each district. 
This meant identifying the initiatives that comprised local accountability systems and 
understanding the local context in which they operate. At the district level, we interviewed 
district administrators and collected documentary materials from the central administration. 

These interviews focused on the kinds of support (curriculum, instructional, assessment, and 
professional development) provided to the schools and how these resources were used. Particular 
attention was paid to how district administrators interpreted and implemented the federal and 
state accountability mandates. We interviewed a total of 20 central office administrators in the 
three cities. Documentary materials included state standards legislation, district reform policies, 
demographic information, district budgets, and school level achievement test scores. 

At the school level, we conducted a total of 196 interviews with principals, teachers, and 
school support faculty over two years. We made two sites visits to each school, one during each 
of the two academic years. Interviews were semi-structured and focused on resource allocation, 
school vision, content and performance standards, curriculum and instructional practices, and 
professional support to teachers. As validity checks on the interview data, we collected 
documentary materials from the schools and district offices. We made 66 classroom observations 
intended to provide a general understanding of the classroom climate and practices. 

To measure teachers’ awareness of state standards’ policy and their perceptions about the 
level of implementation of standards in their school, we asked teachers to complete a survey. 

Two hundred and three teachers from Chicago, seventy-seven teachers from Cleveland, and sixty- 
one teachers from Detroit participated in completing surveys. Two constructs were developed to 
measure teachers’ awareness and perceptions of state policy. The state policy construct included 
8 questions (for example, ‘I refer to the state content and performance standards when assessing 
student performance;’ The programs at my school are evaluated by the state;’ ‘My school 
receives clear and coherent guidance from the state/district to direct school-level reform;’). The 
school standards and practices construct included 11 questions (for example, ‘My school has 
clearly articulated performance goals for all students;’ ‘The curricula in my school are consistent 
with state assessment standards;’ ‘All students in my school have opportunities to learn advanced 
curricula.’). Each question on the teacher survey included a 5-point Likert scale with responses 
ranging from “strongly disagree” to “strongly agree.” The average inter-item reliability 
coefficient was 0.85. Analysis of variance was performed to examine whether there were 
statistically significant district level differences on teachers’ level of awareness of state and 
school district policies and their perceptions of the school standards and practices. Also, bi- 
variate correlations were performed to examine the relationship between teachers’ awareness of 
state policy and perception of school practices. 

The teacher survey also included questions about school climate, instructional strategies, 
and leadership/management practices. School climate included four scales — student-teacher 
relationships, student attitudes, low school problems, and colleague relationships. 
Leadership/management practices included two scales, one on leadership and another on shared 
decision-making. The instructional strategies construct included three scales — student 
evaluation, student-centered instruction, and cooperative learning. Each scale is made up of 
relevant items from the teacher questionnaire. Sample items are listed below. 

School Climate 

• Student- teacher relationships: 4 items (for example, ‘Teachers make students feel 
important;’ ‘Students get along well with teacher.’). 

• Student attitudes: 6 items (for example, ‘Students show pride in and responsibility toward 
the school;’ ‘Students are interested in learning new things.’). 
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• Low school problems; 13 items (for example, ‘Student discipline;’ ‘Students cutting 
classes.’). 

• Colleague relationships: 5 items (for example, ‘Teachers are keen to learn from their 
colleagues;’ ‘I receive encouragement from my colleagues.’). 

Instructional Strategies 

• Student evaluation: 5 items (for example, ‘Completed homework is reviewed and 
discussed in class;’ ‘Student progress is evaluated by daily work samples.’). 

• Student-centered instruction: 5 items (for example, ‘In this class, students work with 
materials that are suited to their own needs;’ ‘When starting a new unit of instruction, 
students use lessons suited to their present abilities.’). 

• Cooperative learning: 4 items (for example, ‘Students in this class work cooperatively to 
achieve goals;’ ‘In this class, students tutor other students.’). 

Leadership/Management 

• Leadership: 4 items (for example, ‘The principal lets staff members know what is 
expected of them;’ ‘The principal talks with teachers frequently about their instructional 
practices.’). 

• Shared decision-making: 3 items (for example, ‘Teachers have significant input in 
decisions about the use of federal resources at my school;’ ‘Teachers are frequently asked 
to participate in decisions concerning administrative policies and procedures.’). 

Policy Context of Standards-Based Reform 
District Accountability Systems 

School district demographic data is summarized in table 3. From the interview data, we 
found both similarities and differences between the three districts regarding the design and 
implementation of standards. We found that standards were one part of an accountability system 
that encompassed state level legislation, regulations, and district reform initiatives. These 
systems were constantly in flux as states and districts developed and changed them in response to 
political and institutional factors. We also found that these districts differed in terms of where 
authority over the accountability system resided. In Chicago, there was strong district level 
accountability; Cleveland was characterized by strong state policies, and Detroit by strong 
school-level autonomy. Each of these policy environments is described below. 

[INSERT TABLE 3 ABOUT HERE] 

Chicago Public Schools: In Chicago, understanding how the accountability system 
works requires taking into account the 1995 Chicago School Reform Amendatory Act (Illinois 
State Assembly, 1995) and the policies put in place by the district under this legislation, 
particularly the retention policy and the probation policy. This legislation, specific only to 
Chicago, gave the district wide discretion to adopt criteria for evaluating schools and students and 
streamlined the procedures for intervening in poorly performing schools (Wong, Dreeben, Lynn, 
& Sunderman, 1997). In response, the district adopted criteria for evaluating schools and 
students that made standardized test scores central to the evaluation process. Schools could be 
placed on “probation” when fewer than 15 % of their students scored at or above the national 
norms on the Iowa Test of Basic Skills (ITBS). Schools were eligible for “reconstitution” if they 
did not improve after being on probation. When a school was reconstituted, the central office 
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appointed a principal and all staff re-applied for their positions. 5 Under this policy, there were 
109 out of 557 on probation for poor academic performance by the 1997-98 school year. Another 
seven schools were reconstituted in June 1997. Two of the four schools we visited in Chicago — 
Cornell and Butler — were on probation. 

The district adopted sanctions for individual students as well. Students could be retained 
in grades 3, 6, 8 and 9 if they did not meet grade level proficiency on standardized tests — the 
ITBS for elementary students or the Test of Achievement and Proficiency (TAP) for ninth 
graders. The district set the required score at approximately one grade level below the national 
norm. This policy also required third, sixth, and eighth graders to have passing grades in reading 
and mathematics and no more than twenty unexcused absences. Ninth graders could have no 
more than twenty unexcused absences and earned at least five credits. While the official policy 
included these additional criteria, in practice, meeting the test score cutoff was the central 
component of the policy and the one schools were most likely to respond to (Roderick, Bryk, 
Jacob, Easton, & Allensworth, 1999). 

Under state legislation, the Illinois State Board of Education established learning goals, 
or standards beginning in 1992. In contrast to the district accountability initiatives, the state 
standards legislation, amended in August 1996, had weak sanctions for not meeting the state 
learning objectives (Illinois School Code, 1998). Schools that were not meeting the standards, as 
measured by the state administered Illinois Goals and Assessment Program (IGAP), could be 
placed on an Academic Watch List by the state 6 When a school was placed on the watch list, 
both the district and school submitted a School Improvement Plan (SIP). Sanctions existed for 
failure to implement an approved SIP and included the possibility of loss of state funds by the 
district or school, the removal of the school board, or the reassignment of students to another 
district or school, actions that were rarely enforced. 

To buttress accountability, the district developed a structure designed to support schools 
placed on probation and help students eligible for retention. It established the Office of 
Accountability to evaluate the strengths and weaknesses of each school, make recommendations 
for improvement, and monitor the improvement process. A probation team was assigned to each 
school to assist the principal and oversee the school improvement process. As part of this 
process, schools on probation were required to work with an external partner chosen by the 
school and contracted to provide educational services to assist the school (Sunderman and 
Nardini, 1999). Title I resources were instrumental in meeting the demands of the district’s 
accountability process. It was, for example, the primary resource available to schools on 
probation to pay the costs of the external partner. 7 

The district accountability support structure also included other strategies designed to 
help students meet the promotional criteria (minimum test score requirements). The district 
provided test preparation materials to help teachers prepare students for the standardized tests, 
implemented “transition classes” for retained students, and supported after school remediation 



5 Beginning in 1999, the district added a process called “reengineering.” This is a step between probation 
and reconstitution adopted to help schools that are not improving on probation, but intended to avoid the 
extreme measure of reconstitution. 

6 Beginning in the spring of 1999, the state introduced a new state test, the Illinois Standards Achievement 
Test (IS AT) that replaced the IGAP. 

7 The district office paid the costs of the external partnership during the first year of probation or 
reconstitution. During the second year, the cost was split evenly between the schools and the Office of 
Accountability, while the third year was paid entirely by the schools. 
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programs for students needing extra help in meeting the test score cutoff point. Students who did 
not meet the promotion criteria were required to attend summer school, where they received 
instruction on a specially designed curriculum to help them improve test scores and be promoted 
to the next grade. 

In contrast to the resources devoted to the probation and retention policies, the district 
provided minimal support to schools to implement standards. To comply with the requirements 
of the state’s standards legislation, the district developed the Chicago Academic Standards and 
Curriculum Framework Statements (Chicago School Reform Board of Trustees, 1997). However, 
they left it up to the schools and/or individual teachers to align the curriculum with the standards 
and adapt instructional materials to meet the needs of the students being served. 

Cleveland Public Schools: The accountability system that emerged in Cleveland was 
shaped largely by state legislation establishing proficiency standards and resulted in a system 
driven by the improvement of standardized achievement scores. Legislation enacted by the Ohio 
General Assembly in August 1997 established a rating system that outlined minimum 
performance standards that each district must meet (Ohio General Assembly, 1997). These 
included minimum scores on the Ohio Proficiency Exam, a minimum student attendance rate (93 
%), and a maximum dropout rate (3 %). Districts not meeting these criteria were required to 
develop a three-year “continuous improvement plan” and were subject to intervention under rules 
developed by the state board of education. Stringent academic requirements existed at the student 
level as well. The bill established a high school exit exam and increased the credit hours required 
for graduation. To improve reading scores of elementary students, the bill contained a “fourth 
grade guarantee,” that is, fourth grade students who failed to pass the reading portion of the Ohio 
Fourth-Grade Proficiency Test would be retained beginning in the 2001-02 school year. 

Retention was extended to include students who were truant and failed two or more subjects. 

In response to the state legislation, the Cleveland district established an accountability 
system where student outcomes, particularly outcomes on standardized achievement tests, were 
used as the means of evaluating schools. “Continuous improvement” was a hallmark of this 
system. Two performance measures were identified: improving student achievement as 
measured on the Ohio Proficiency tests and improving student attendance at school (Cleveland 
City Public Schools, 1997). School improvement “targets” were identified for each school that 
took into account the past performance of the school’s population. For example, Lincoln, where 
16.2 % of the fourth grade cohort passed the fourth grade Ohio proficiency in 1996-97, had a 
target of 21.2 % passing in 1997-98. 8 The attendance rate in 1996-97 was 91.0 %; the target for 
1997-98 was 91.7 %. A third objective, improving the organizational efficiency and school 
climate, was measured through a school survey. According to one district administrator, 
“Improved student performance on the Ohio proficiency, of course, is the real bottom line.” With 
this goal, the system put pressure on both schools and individual students to improve test scores. 



8 Using a regression analysis, the district determined a predicted level of performance on the proficiency 
test based on students’ performance on CAT reading tests, the schools’ mobility rate, and poverty index. 
The predicted level of performance was then compared to the observed level of performance, and a target 
was established for the following year. Schools that performed at much higher than predicted rates were 
given lower improvement targets, and schools with much lower than expected performance were given 
higher improvement targets. Schools were evaluated only on those students who had been in the school 
from early October through the spring testing week. For more information, see Cleveland City Public 
Schools, 1997. 
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With accountability tied to the Ohio Proficiency test, many of the schools in Cleveland 
were focusing on improving literacy skills as measured by the proficiency test. To help schools 
meet these skills, the district adopted a new reading series that was more closely aligned with the 
state standards. The district also provided a reading resource teacher to the schools for the 
primary grades (K-2). In January 1999, the new CEO introduced “warm up” activities in all the 
elementary schools (grades 3 through 8). These were exercises in the core curriculum areas for 
students to do for ten minutes at the start of each school day. They were intended to assist 
students in preparing for the Ohio Proficiency test. 

The district also began to administer two assessments in addition to the Ohio Proficiency 
test. One was the off-grade proficiency, administered in grades one through three. This tested 
students on the same areas as the Ohio proficiency and was intended to help schools track student 
progress. The goal was to identify students who were not doing well before they reached the 
fourth grade. The other test was an interim test for students in grades three through eight in 
reading and math. The data from this test were available within two weeks of the testing, “so two 
weeks after taking the test every classroom teacher knows what their youngsters did and knows 
their area of weakness.” 9 The goal was to insure that students were performing at grade level. 
Commenting on the district testing policy, one administrator said: 

“If you have this kind of data for every kid in your class who was present and took the 
test, nobody should fall through the cracks. No one should. If you had this kind of data 
at grades one, two, and three and you know the high stakes test is grade four, then you 
begin to use this data to provide support and to focus everyone on what it is you have to 
do. You miss not a single youngster.” 10 

In 1997, Ohio passed legislation similar to that in Illinois that changed the governance 
structure of the district. As in Illinois, this legislation gave the mayor the authority to appoint the 
CEO. The CEO and school board were given the authority to take corrective actions in schools 
where students were not performing at an acceptable level. In Cleveland, this included the 
authority to reallocate academic and financial resources, reassign staff, and redesign of the 
academic program (Ohio General Assembly, 1998). Implementation of this legislation was 
delayed by a court challenge until fall 1998. When we conducted our district office interviews in 
February 1999, the new CEO, appointed by the mayor under provisions of this legislation, had 
been on the job for just 1 1 weeks. This new legislation came two years after the state declared 
Cleveland schools to be in a “state of crisis” and took control of the district. 

Detroit Public Schools: The system that emerged in Detroit left schools with 
considerable discretion to interpret and implement standards-based reforms. Three factors — state 
tradition of local control, state policies that contributed to district office fragmentation, and strong 
site-based management — were instrumental in shaping an accountability system in Detroit that 
left considerable discretion to school administrators and teachers, while limiting the authority of 
the district office. 

The state standards legislation, enacted in 1990, required schools to report educational 
progress to the public and included a school accreditation process that took into account student 
achievement on the state assessment. 11 However, the Michigan law left considerable discretion to 



9 District Office Administrator interview, Cleveland City Schools, February 12, 1999. 

|°Ibid. 

Michigan enacted legislation in 1990 mandating the development of a district and school improvement 
plan, a core curriculum in every district, an annual educational report to the public for every school, and a 
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the districts. For example, the state provided no description of the criteria for identifying school 
districts in need of improvement (Yu and Taylor, 1999) and had weak sanctions for schools or 
districts not meeting the performance standards. Sanctions existed for poor-performing schools, 
and included replacing the building administrator, provisions for parents to choose any accredited 
school within the district, and state take-over and closure, actions that were rarely undertaken. In 
1 997, the State Board of Education approved a revised definition of adequate yearly progress for 
Title I schools that included identifying the percent of students meeting state expectations on the 
Michigan Educational Assessment Program (MEAP). This revision retained local discretion by 
allowing schools to establish their own targets for improvement and giving them two years to 
meet their achievement goals before being placed on school improvement status. 12 

Teachers retained considerable discretion within the Detroit accountability model as well. 
To determine how well students were meeting the standards, the district adopted exit skills for 
grades K-5 that were aligned with the state performance standards and outlined what a student 
should know at each grade level (implementation began in 1998). Students who had not mastered 
the exit skills by the end of the school year could be retained. Implementation of the exit skills 
depended on teacher and principal discretion. Teachers assessed students on the exit skills 
throughout the school year, noting when they mastered a particular skill. Teachers and principals 
were responsible for final decisions regarding promotion. 

School autonomy was reinforced by frequent district reorganization and downsizing and 
political attempts to take over the district. Under pressure from the state, the district undertook a 
number of steps to reform district administration. In September 1997, the board adopted a re- 
organization plan that included eliminating the six area offices. This was intended to streamline 
the central administration and save money. As part of the re-organization, the board voted in 
April 1998 to eliminate 15 out of 45 administrative positions, including five area superintendents. 
In November 1997, the board ousted then Superintendent David Sneed and replaced him with 
Eddie Green, a long time Detroit school administrator. 

At the same time, efforts by the state to pass legislation that would give the state the 
authority to take over the district were under way. Governor Engler first introduced legislation 
that would give the state authority to take over low performing school districts in 1997. The plan 
failed to pass the legislature where the Democrats held a majority in the House and some 
Republicans opposed the plan in deference to maintaining local control. In January 1999, 
Governor Engler again proposed legislation that would allow a state takeover of a district school 
system. This time the legislation included only the Detroit district. This legislation, modeled on 
similar reform plans in Chicago, Baltimore, and Cleveland, put the mayor in charge of the 
schools. The bill passed the Republican dominated legislature in March 1999 and was quickly 
signed into law by the governor. 13 Following passage of the legislation, the mayor appointed 
Eddie Green as acting CEO. David Adamany assumed the position in May 1999. 

This focus on governance change fostered a sense of discontinuity in district leadership, 
contributed to fragmentation in district policy, and strengthened the autonomy of local schools. It 
was not unusual for each change in administration to be accompanied by organizational and 
programmatic changes. For example, a district-wide “cluster” organization was replaced with 



school accreditation process that took into account student achievement on the state assessment (Act No. 
25, March 13, 1990, State of Michigan 85 th Legislature, Regular Session of 1990). 

12 Interview with Michigan State Department of Education administrator, March 24,2000. 

13 The House approved the bill, 66-43, and the Senate approved the House version of the bill, 31-7 (Darci 
McConnell and Chris Christoff, 1999). 
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“constellations” following one such reorganization. For one school we visited, this 
reorganization put the middle school that the elementary students attended in a different 
constellation. Under the reorganization that put the mayor in charge of the schools, a new 
promotion and retention plan was adopted that co-existed with the exit skills program. 

Frequent leadership change also reinforced school level autonomy. Until they were 
eliminated, the area offices retained considerable authority over the schools in their area and the 
staffs in their office. 14 The elimination of the area offices and a reduction in the central office 
bureaucracy shifted decision-making up, with many decisions, even minor ones, made by the 
superintendent. The elimination of area offices further weakened the link between the schools 
and the administration. Without area offices, principals reported directly to the CEO. Title I 
administrators, who monitored schools for compliance, could make recommendations to the 
school and superintendent but had no authority to intervene in a school. 15 At the school level, 
much of the organizational change that accompanied the governance reform of the Detroit central 
office was decoupled from the day-to-day operations of the school. School administrators and 
teachers took a “wait and see” attitude towards district policies, while continuing their daily 
routines. This was reinforced by site-based management, which gave schools authority over 
program decisions at the school level. It was unclear at the time of this study whether the 
governance reforms enacted in 1999 would strengthen the role of the central office or contribute 
to continued fragmentation in district policy. 

In sum, the accountability policies adopted by the three states we visited — Illinois, Ohio, 
and Michigan — were characterized by the adoption of policies where student outcomes, 
particularly outcomes on standardized achievement tests, were used as the means of evaluating 
schools. In Chicago, the district exerted considerable control over the design and implementation 
of accountability policies. This was facilitated by the adoption of governance reforms by the state 
legislature in 1995 that expanded and strengthened the managerial and administrative role of the 
district. Under this legislation, the district was given increased authority and the flexibility to 
intervene in low performing schools. In Cleveland, the state accountability policies formed the 
bases for the district’s policies. Based on state guidelines, school improvement targets were 
identified for each school that took into account the past performance of the school’s population. 
The system put pressure on both schools and individual students to improve their performance on 
the Ohio Proficiency exam. In contrast, in Detroit the design of the state standards’ legislation 
emphasized local discretion and a focus on governance change fostered a sense of discontinuity in 
district leadership, contributed to fragmentation in district policy, and reinforced the autonomy of 
local schools. 



Survey Results 



School Level Implementation 



From the teacher survey data, we found statistically significant differences between 
districts in teachers’ awareness of state policy on standards and their perceptions of how well 
schools were implementing them. As summarized in table 4, teachers from Chicago expressed 
the highest level of awareness of the state’s policy on standards (range 1-5, mean 3). Detroit and 
Cleveland were very similar in the level of teachers’ awareness of state standards. On the school 
standards and practices construct, teachers’ perceptions of how well schools were implementing 
standards were highest in Chicago, followed by Detroit. Teachers’ perceptions about the 
implementation of standards were lowest in Cleveland. There is a high correlation between 
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District office interview, Detroit Public Schools, September 24, 1999. 
Ibid. 
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awareness of state standard policies and perceptions of school practices (.68), which indicates that 
the more teachers were aware of the policies, the higher their perception that standards were 
being implemented. 



[INSERT TABLE 4 ABOUT HERE] 

We also found statistically significant differences among districts on three of the scales 
measuring school climate (table 4). Student-teacher relationships, student attitudes, and lack of 
school problems were significantly different between districts while teacher relationships were 
not. There were also differences between districts on leadership, but not on shared decision 
making. Interestingly, teachers from Cleveland consistently rated lower on all of these scales 
than teachers from Chicago or Detroit, and teachers from Detroit rated the three school climate 
scales higher than teachers from Chicago or Cleveland did. None of the constructs measuring 
instructional strategies were significant, although the direction of these means also showed 
Cleveland lower than Chicago or Detroit. 

To account for the differences between districts in the level of implementation of 
standards, we examined the differences in policy environments between the districts. While we 
interpret the survey findings cautiously, these findings suggest that differing policy environments 
are related to teachers’ awareness of standards policies, their perceptions of how well they are 
implemented in the school, and their perceptions of school climate. The literature on policy 
implementation is useful in understanding these differences. It suggests that in the first few years 
of a program’s existence, there is more likely to be uncertainty over how the program will operate 
and higher levels of conflict between different levels of the educational system, especially when 
new programs differ from established practice (Peterson, Rabe, & Wong, 1986). Programs 
adjustments are a common feature in the implementation of new programs. Over time, 
regulations and guidelines are clarified and objectives are modified as feedback and evaluations 
provide information on how well the programs operate. Practitioners familiarize themselves with 
the policies and programs, make adjustments to accommodate the new demands, and translate 
their understanding of the policies into practice (Lipsky, 1980). 

The level of the educational system (state, district, or school) where authority over policy 
is located is also likely to be related to how well policies are implemented since different levels of 
the educational system are likely to have different concerns and tools available to them to shape 
practice (Wong & Sunderman, 1996). In the case of standards, political leaders have become 
more directly involved on issues traditionally decided by professionals. The tools available to 
them to influence practice include regulations and output controls (e.g., assessments) whereas 
teachers are likely to use their professional judgment and discretion (Rowan, 1991). 

In Chicago where there was a well-articulated district level accountability system with 
strong sanctions, teachers were well aware of the policies and perceived that they were 
implementing them. This is in contrast to Cleveland where the district role was subordinate to 
that of the state and individual schools had relatively little control over standards and 
accountability policies. The legislature had recently enacted an accountability system when we 
surveyed the teachers, contributing to the uncertainty among teachers that was reflected in their 
responses on the survey. For example, many of the teachers we interviewed in Cleveland 
expressed concern about the 4 lh grade retention policy, even though it had not yet been 
implemented. They were uncertain about how best to implement the policy and the implication 
of the policy on students in earlier grades who had not learned what they should. They struggled 
with whether or not l sl or 2 nd grade students should be retained or promoted in hopes that they 
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would catch up . 16 Many also felt establishing performance criteria based on what a particular 
grade level achieved in the prior year (past performance) was unfair since it did not take into 
account individual differences in learning. 

In Detroit, the relative autonomy of teachers at the school level may have contributed to 
the lower level of awareness of and implementation of standards as well as the more positive 
responses on the school climate scales. Their relatively high degree of autonomy from 
organizational authority gave them considerable discretion to interpret and modify policies and 
programs to fit existing practices. For example, as already mentioned, implementation of the exit 
skills and decisions about whether or not to retain students depended entirely on principal and 
teacher discretion. Moreover, teachers did not view the exit skills as that much different from 
many of the assessment practices they were already using. They also considered the standards as 
simply a revision of an earlier curriculum framework that had been in place, and there were no 
consequences attached for not following the standards other than what the principal might choose 
to do. 



Teachers’ awareness of state standards does not seem to be related to an emphasis on 
certain instructional strategies or practices (i.e., student evaluation, student centered instruction, 
or cooperative learning). The failure to find differences on instructional strategies between 
districts suggests that schools may not be altering their practices in ways envisioned by reformers. 
In the case studies, we found that schools were likely to emphasize discrete aspects of the 
instructional program rather than attend to instructional and curricular issues. Schools, 
particularly the lower performing schools, adopted add-on programs or targeted programs to meet 
the demands of the accountability system, particularly the improvement of test scores. Two 
strategies that schools used included allocating resources to reduce class size and targeting 
particular grades, subjects, and/or students for remediation. There was some evidence that higher 
performing schools were better able to address instructional issues. These schools were more 
likely to increase the amount of instructional time that students received, thereby increasing the 
amount of learning time and content coverage necessary to improve student achievement. 

Conclusions and Implications 

The assumption behind the adoption of standards, curriculum frameworks, and 
assessments is that they will alter teaching practice and improve learning. In this paper we 
examined the implementation of standards-based reforms in three urban districts to understand 
the influence of state policy on school practices. Our findings suggest that the different policy 
environments within which schools and districts operate is related to how districts and school 
interpret standards. These differences translated into differences in teachers’ awareness of 
standards and perceptions of how well they were implemented. Also, we found from surveys 
that while teachers reported a high level of awareness of standards, the interviews indicated that 
they were not the driving force behind school practices — teachers weren’t necessarily using them 
to guide classroom practice. Instead, we found district accountability policies that were narrowly 
focused on raising standardized test scores and, not surprisingly, school practice directed at 
improving the school’s standardized test scores. 

This study also found that it is difficult if not impossible to examine standards-based 
reform without taking into account other accountability and governance reforms. The policies are 
interrelated, with one policy establishing the context for another (such as governance reform in 
Chicago establishing the preconditions for the district’s accountability policies). The 



16 Principal interview, Lincoln Elementary School, Detroit Public Schools, 3-4-98. 




13 



ii 



implementation of standards-based reform also involved actors at multiple levels of the 
educational system with different levels of authority. While there were differences depending on 
the local context, standards-based reform relied on top-down systems where student performance 
on standardized achievement tests was the driving force. This is not surprising since outcome 
controls are a primary tool available when authority is centralized to create pressure and place 
constraints on schools. 

To conclude, it is unclear if standards-based reform is moving in a direction that will 
bring increased policy coherence and improved teaching and learning. While the goal is to 
improve student achievement, it begs the question of what are the consequences of standards- 
based reform for teaching and learning. By focusing on teaching as a set of classroom activities 
that can be changed with the right set of incentives and sanctions, questions about the legitimacy 
of teaching arise when schools are not able to deliver acceptable levels of learning. Clearly, more 
research is needed on the impact of standards-based reform on the curriculum students receive 
and instructional practices used by teachers. Districts can assist schools by establishing an 
infrastructure that provides much needed information on developments in curriculum, instruction, 
and assessment practices. More attention needs to be devoted to helping teachers align the 
curriculum and their instructional practices with the standards, and less to test preparation. 
Policymakers need to understand the implications of standards for teaching and learning as they 
move forward with accountability policies. 
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Table 1: Summary of Observed and Expeeted Achievement Scores and the Residual for 
Students at Selected Title I Schoolwide Programs in Three Districts: 

Chicago, Cleveland, and Detroit 



Chicago Public Schools, Iowa Tests of Basic Skills, 1990-91 to 1995-96 



School 


Observed 
( average ) 


Reading 

Expected 


Residual 


Observed 

(average) 


Math 

Expected 


Residual 


Fairfax 


15.99 


14.61 


1.37 


27.84 


16.94 


10.93 


Montgomery 


23.27 


14.14 


9.13 


28.07 


21.17 


6.94 


Cornell 


11.87 


20.27 


-8.40 


15.57 


22.49 


-6.89 


Butler 


11.69 


16.87 


-5.19 


9.90 


18.77 


-8.85 



Source: William L. Yancey & Raj Thadani (1997). Identifying Exceptional Schools: Chicago , 
Illinois. Laboratory for Student Success, Center for Research on Human Development and 
Education, Temple University. 

Cleveland Public Schools, California Achievement Test, 1994-95 & 1995-96 



Reading 

Observed Expected Residual 

School 1995 1996 1996 1995-96 



Sherman 


49.5 


47.9 


42.8 


5.6 


Lincoln 


52.0 


53.0 


43.9 


8.0 


Carter 


36.7 


36.9 


42.2 


-6.3 


McKinley 


39.1 


36.7 


42.1 


-4.9 



Source: William L. Yancey & Raj Thadani (1997). Identifying Exceptional Schools in Cleveland. 
Laboratory for Student Success, Center for Research on Human Development and Education, 
Temple University. 



Detroit Public Schools, Metropolitan Achievement Tests, 1995-96 



School 


Observed 


Reading 

Expected 


Residual 


Observed 


Math 

Expected 


Residual 


Washington 


58 


39.4 


18.6 


63 


38.0 


25.0 


Sinclair 


67 


39.1 


27.9 


59 


37.8 


21.2 


Adams 


14 


44.6 


-30.6 


16 


42.3 


-26.3 


Roosevelt 


15 


36.6 


-21.6 


21 


36.7 


-15.7 



Source: Personnel communication with William L. Yancey, Temple University (September 24, 
1997), “Results of Multiple Regression Analyses of Reading and Math Achievement (MAT) 
Detroit Elementary and Middle Schools, 1996.” 
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Table 2: Demographic and School Characteristics for Students at Selected Schools in Three 
Districts: Chicago, Cleveland, and Detroit 

Chicago Public Schools, 1996-97 





Fairfax 


Montgomery 


Cornell 


Butler 


Enrollment 


441 


956 


667 


494 


% White 


0.0 


0.8 


0.0 


0.0 


% African Amer. 


100 


0.1 


100 


100 


% Latino 


0.0 


99.0 


0.0 


0.0 


% Asian 


0.0 


0.1 


0.0 


0.0 


% Native American 


0.0 


0.0 


0.0 


0.0 


% LEP 


0.0 


59.0 


0.0 


0.0 


% Low Income 


95.5 


98.1 


86.7 


99.8 


% Mobility 


34.1 


24.1 


24.5 


32.3 


% Daily Attend. 


92.7 


94.2 


91.7 


91.9 


Source: Chicago Public Schools 



Cleveland Public Schools, 1996-97 

Carter 


McKinley 


Sherman 


Lincoln 


Enrollment 


320 


516 


616 


562 


% White 


1.4 


2.2 


37.2 


1.8 


% African American 


97.2 


97.0 


18.0 


95.8 


% Latino 


0.8 


0.2 


43.4 


2.3 


% Asian 


0.0 


0.2 


0.2 


0.0 


% Native American 


0.6 


0.0 


0.5 


0.0 


% LEP 


0.0 


0.0 


25.4 


0.0 


% Low Income 


96.2 


95.0 


95.8 


96.3 


% Mobility 


32.5 


29.0 


22.6 


16.7 


% Daily Attend. 


91.6 


89.2 


91.3 


91.0 


Source: Cleveland City School District, Building Profiles: Data for 1996-97 School Year , Three- 
Year Baseline Data , Elementary Schools , October 1997. *Elementary School Average. N/A: not 
available. 



Detroit Public Schools, 1996-97 








Adams 


Roosevelt 


Washington 


Sinclair 


Enrollment 460 


728 


564 


781 


% African Amer. 18.0 


78.0 


95.0 


100 


% Low Income 71 .0 


96.0 


80.0 


68.0 



Source: Detroit Public Schools 
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Table 3: School District Demographics, 1996-97 





Chicago 


Cleveland 


Detroit 


Enrollment 


421,334 


71,054 


183,447 


% African Amer. 


54.7 


79.5 


90.98 


% Latino 


30.6 


7.5 




% Low Income 


83.2 


86.4 


69.87 


% Mobility 


29.0 


19.6 





Source: School district database. 



Table 4: ANOVA 





Chicago 


Cleveland 


Detroit 


ANOVA 


Constructs 


Mean (SD) 


Mean (SD) 


Mean (SD) 


F(df) 



Standards Awareness 



State Policy 


4.06 


(.51) 


3.79 (.50) 


3.76 (.59) 


11.95 (2, 338)** 


Standards 


4.00 


(.60) 


3.67 (.63) 


3.85 (.69) 


6.77 (2, 338)** 


School Climate 


Student-Teacher Rel 


4.24 


(.56) 


4.06 (.49) 


4.34 (.59) 


4.76 (2,326)** 


Student Attitude 


3.91 


(.72) 


3.60 (.67) 


3.99 (.68) 


6.43 (2, 320)** 


Lack of Problem 


3.15 


(.64) 


2.92 (.45) 


3.29 (.36) 


7.38 (2, 306)** 


Colleague Relationship 


4.04 


(.64) 


3.98 (.74) 


4.13 (.67) 


0.76 (2, 322) 


Leadership/Management 
Leadership 4.05 


(.84) 


3.75 (.91) 


3.86 (.89) 


2.86 (2, 332)* 


Shared Decision 


3.39 


(.93) 


3.12(1.04) 


3.41 (.91) 


2.33 (2, 321) 


Instructional Strategies 


Evaluation 


4.15 


(.57) 


4.04 (.52) 


4.11 (.57) 


0.90 (2, 304) 


Student Centered Inst 


3.98 


(.57) 


4.01 (.61) 


3.92 (.69) 


0.32 (2, 313) 


Cooperative Learning 


4.23 


(.65) 


4.161.58) 


4.33 (.56) 


1.24(2. 327) 


Note: ***<.000 


** <.01 *<.05 
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